A Brief Introduction on Shannon's Information Theory
نویسنده
چکیده
This is an introduction to Shannon’s Information Theory. It is more like a long note so that it is by no means a complete survey or completely mathematically rigorous. It covers two main topics: entropy and channel capacity. All related concepts will be developed in a totally combinatorial flavor. Some issues usually not addressed in the literature will be discussed here as well. Hopefully, it will be interesting to those interested in Information Theory.
منابع مشابه
Psychometric Properties of the Brief Form of Professor-Students Rapport Scale-based on Classical Test Theory and Item-Response Theory
Introduction: In order to improve the quality of the teaching process, it is necessary to review the professor-student rapport. The purpose of the present study was to investigate the factor structure and item-response parameters of Professor-Students Rapport Scale-Brief (PSRS-B). Methods: In a descriptive-correlation study, 497 students from Shahid Beheshti University of Medical Sciences were ...
متن کاملThe Role of Information Theory in Cryptography
This paper reviews the relations between information theory and cryptography, from Shannon's foundation of information theory to the most recent developments in unconditionally-secure key-agreement protocols. For a long time, information theory has mainly been used in cryptography to prove lower bounds on the size of the secret key required to achieve a certain level of security in secrecy and ...
متن کاملClaude Shannon : Biologist :
Claude Shannon founded information theory in the 1940s. The theory has long been known to be closely related to thermodynamics and physics through the similarity of Shannon's uncertainty measure to the entropy function. Recent work using information theory to understand molecular biology has unearthed a curious fact: Shannon's channel capacity theorem only applies to living organisms and their ...
متن کاملTheory CMU : Spring 2010 Notes 1 : Introduction , linear codes
The theory of error-correcting codes and more broadly, information theory, originated in Claude Shannon’s monumental work A mathematical theory of communication, published over 60 years ago in 1948. Shannon’s work gave a precise measure of the information content in the output of a random source in terms of its entropy. The noiseless coding theorem or the source coding theorem informally states...
متن کاملA brief review of molecular information theory
The idea that we could build molecular communications systems can be advanced by investigating how actual molecules from living organisms function. Information theory provides tools for such an investigation. This review describes how we can compute the average information in the DNA binding sites of any genetic control protein and how this can be extended to analyze its individual sites. A for...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
- CoRR
دوره abs/1612.09316 شماره
صفحات -
تاریخ انتشار 2016